Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0 #5097

Merged
merged 1 commit into from
Mar 15, 2023
Merged

HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0 #5097

merged 1 commit into from
Mar 15, 2023

Conversation

haohao0103
Copy link
Contributor

Based on branch-2,HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0.

JIRAID:HBASE-27688
3797df1 
HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0
@Apache9 Apache9 changed the title JIRAID:HBASE-27688 HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0 Mar 13, 2023
@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 1s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚 mvninstall 3m 35s branch-2 passed
+1 💚 compile 2m 22s branch-2 passed
+1 💚 checkstyle 0m 34s branch-2 passed
+1 💚 spotless 0m 43s branch has no errors when running spotless:check.
+1 💚 spotbugs 1m 30s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 21s the patch passed
+1 💚 compile 2m 21s the patch passed
+1 💚 javac 2m 21s the patch passed
+1 💚 checkstyle 0m 36s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 17m 54s Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.4.
+1 💚 spotless 0m 41s patch has no errors when running spotless:check.
+1 💚 spotbugs 1m 34s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 12s The patch does not generate ASF License warnings.
38m 2s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #5097
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname Linux 354b709c8272 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 0f67503
Default Java Eclipse Adoptium-11.0.17+8
Max. process+thread count 86 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 43s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 4m 17s branch-2 passed
+1 💚 compile 0m 45s branch-2 passed
+1 💚 shadedjars 5m 2s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 27s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 40s the patch passed
+1 💚 compile 0m 46s the patch passed
+1 💚 javac 0m 46s the patch passed
+1 💚 shadedjars 4m 59s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 24s the patch passed
_ Other Tests _
+1 💚 unit 196m 33s hbase-server in the patch passed.
221m 36s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #5097
Optional Tests javac javadoc unit shadedjars compile
uname Linux 192710e48fe7 5.4.0-1097-aws #105~18.04.1-Ubuntu SMP Mon Feb 13 17:50:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 0f67503
Default Java Eclipse Adoptium-11.0.17+8
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/testReport/
Max. process+thread count 2821 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 37s Docker mode activated.
-0 ⚠️ yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 2m 59s branch-2 passed
+1 💚 compile 0m 34s branch-2 passed
+1 💚 shadedjars 4m 26s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 23s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 46s the patch passed
+1 💚 compile 0m 35s the patch passed
+1 💚 javac 0m 35s the patch passed
+1 💚 shadedjars 4m 24s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 21s the patch passed
_ Other Tests _
-1 ❌ unit 208m 49s hbase-server in the patch failed.
230m 6s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #5097
Optional Tests javac javadoc unit shadedjars compile
uname Linux df65750736d7 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 0f67503
Default Java Temurin-1.8.0_352-b08
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/testReport/
Max. process+thread count 2607 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5097/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@haohao0103
Copy link
Contributor Author

@Apache9 hello ,I have checked the error log, the error content is mainly the connection timeout, I am at a loss to understand this error, could you please guide me? Thank you. The error stack is as follows:

org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testLoading[0: serialPeer=true]

Error Message
Failed to get result within timeout, timeout=60000ms
Stacktrace
java.io.IOException: Failed to get result within timeout, timeout=60000ms
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:242)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:53)
at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:191)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:271)
at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:440)
at org.apache.hadoop.hbase.client.ClientScanner.nextWithSyncCache(ClientScanner.java:314)
at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:612)
at org.apache.hadoop.hbase.client.ResultScanner.next(ResultScanner.java:95)

@Apache9
Copy link
Contributor

Apache9 commented Mar 13, 2023

2023-03-13T04:16:20,762 ERROR [RS_REFRESH_PEER-regionserver/df65750736d7:0-0.replicationSource,2.replicationSource.wal-reader.df65750736d7%2C35391%2C1678680878451,2] regionserver.HRegionServer(2747): ***** ABORTING region server df65750736d7,35391,1678680878451: Unexpected exception in RS_REFRESH_PEER-regionserver/df65750736d7:0-0.replicationSource,2.replicationSource.wal-reader.df65750736d7%2C35391%2C1678680878451,2 *****
java.lang.NullPointerException: null
at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.switched(ReplicationSourceWALReader.java:224) ~[classes/:?]
at org.apache.hadoop.hbase.replication.regionserver.SerialReplicationSourceWALReader.readWALEntries(SerialReplicationSourceWALReader.java:114) ~[classes/:?]
at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:178) ~[classes/:?]
at org.apache.hadoop.hbase.replication.regionserver.SerialReplicationSourceWALReader.run(SerialReplicationSourceWALReader.java:35) ~[classes/:?]

Should not be your problem. Likely a bug introduced by HBASE-27632.

Will merge this and file an new issue to solve the problem.

@haohao0103
Copy link
Contributor Author

2023-03-13T04:16:20,762 ERROR [RS_REFRESH_PEER-regionserver/df65750736d7:0-0.replicationSource,2.replicationSource.wal-reader.df65750736d7%2C35391%2C1678680878451,2] regionserver.HRegionServer(2747): ***** ABORTING region server df65750736d7,35391,1678680878451: Unexpected exception in RS_REFRESH_PEER-regionserver/df65750736d7:0-0.replicationSource,2.replicationSource.wal-reader.df65750736d7%2C35391%2C1678680878451,2 *****
java.lang.NullPointerException: null
at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.switched(ReplicationSourceWALReader.java:224) ~[classes/:?]
at org.apache.hadoop.hbase.replication.regionserver.SerialReplicationSourceWALReader.readWALEntries(SerialReplicationSourceWALReader.java:114) ~[classes/:?]
at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:178) ~[classes/:?]
at org.apache.hadoop.hbase.replication.regionserver.SerialReplicationSourceWALReader.run(SerialReplicationSourceWALReader.java:35) ~[classes/:?]

Should not be your problem. Likely a bug introduced by HBASE-27632.

Will merge this and file an new issue to solve the problem.

ok,i got it .thanks

@Apache9 Apache9 merged commit 6920e72 into apache:branch-2 Mar 15, 2023
Apache9 pushed a commit that referenced this pull request Mar 15, 2023
@haohao0103 @Apache9
HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_T…
52d3824 
…S of hfileinfo is 0 (#5097)

Co-authored-by: alanzhao <alanzhao@126.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6920e72)
Apache9 pushed a commit that referenced this pull request Mar 15, 2023
@haohao0103 @Apache9
HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_T…
becbc3f 
…S of hfileinfo is 0 (#5097)

Co-authored-by: alanzhao <alanzhao@126.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6920e72)
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
@haohao0103 @Apache9
HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_T…
eb140f6 
…S of hfileinfo is 0 (apache#5097)

Co-authored-by: alanzhao <alanzhao@126.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6920e72)
(cherry picked from commit 52d3824)
Change-Id: I4bbfb1acd2e7fafc6f15c0562bcee581e24ac81f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Apache9 Apache9 Apache9 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@haohao0103 @Apache-HBase @Apache9